Communications - A Technical Overview
By Michael Hickey
Copyright (c) 1993 Apple Users' Group, Sydney
Republished from Applecations, a publication of the Apple Users' Group, Sydney, Australia.


INTRODUCTION
In the last decade there has been an enormous growth in the popularity of personal data communications using the public telephone network by Australian computer users. This popularity has come about through the reduction of the price and improvement in quality of modems for business use and the increase in on-line services available to subscribers. This, in its turn, has caused an upswelling of low cost, computer based communications technologies into the personal computer user markets. Not three years ago, modems capable of 9600 bps (bits per second) cost over $1800, today these modems cost as low as $400, well within the reach of most hobbyists.
This article is an attempt to describe, in moderate detail, what happens when two modems communicate over a telephone line. I assume that you are already moderately familiar with how data is represented in a computer as binary data.

Before I launch into the article proper, I shall first take you off on a short tangent to discuss some of the essential problems with using the telephone network to transmit computer data.

LIMITATIONS OF THE TELEPHONE NETWORK
The human ear detects sound waves, the vibration of air molecules causing the next to vibrate in a wave form which can be likened to the ripples on a pond. Because of its basic design, the human ear does not hear every single sound generated by the world around us. By design, or accident, the human ear "samples" just a small part of the world of sound around us, typically capable of detecting frequencies up to 20Ê000 Hz.
Strangely, the human voice, however, is capable only of producing sounds in the range 30 Hz to 10Ê000 Hz.   The limit on our human "communications channel" is, therefore, not the ear as some may expect, but the range of sounds the human voice can produce. Fortunately, for normal speech, we only produce sounds within the 300 Hz and 3400 Hz frequencies.
It is for this reason that the traditional public telephone network had been designed to transmit electronically, signals between 300 Hz and 3400 Hz. This is sufficient to carry voice in an intelligible form over great distances. Unfortunately, due to the fundamental nature of twisted pairs of copper wire used in the classic Telecom line, the characteristics of the line varied over the transmission band causing distortion to signals comprising more than one frequency.
Of course use of optic-fibre and digital transmission have largely removed the problem on these lines but many thousands of kilometers of twisted pair lines still remain to distort your signal. There are two types of distortion involved: attenuation distortion and delay distortion.
Attenuation distortion is caused by a loss of signal strength across different frequencies at different rates. Telecom has installed repeaters every two kilometers along the twisted pair lines which boost the signal in such a way to compensate for the frequencies most likely to be affected by attenuation.
Delay distortion occurs when frequencies arrive at different moments even though they have been transmitted at the same time. This is a problem only if the frequencies need to be synchronised and in data communications they would have to be synchronised. Delay distortion is overcome by the use of equalisers to give an approximately equal delay for all frequencies transmitted over the 300 Hz to 3400 Hz band. The process by which repeaters and equalisers are added to a line is called conditioning, and results in the frequency strengths (amplitudes) lining up in a curve similar to the one I have shown in Figure 1.

These problems are gradually being fixed by the installation of optical-grade glass-fibre, which carries digital signals as discrete pulses of light. The problems will not be completely overcome until twisted-pair copper wire is completely phased out over the next decade.

BINARY DATA SIGNALS
In computers, data is represented in a form unlike normal analog (voice) signals. Instead of rising and falling like the traditional frequency wave, the digital data signal is binary, (that is, it consists of only two distinct states such as high and low or on and off) and consists of rectangular pulses.
Inside a computer, the data lines are maintained at a constant voltage to represent the '0' in a binary signal and is 'pulled' to another voltage to represent the '1', something like a DC electric current, where the power is either on or off. The '1' in a binary signal would be where the power is on.
It is possible to simulate a pulse by using an analog square wave, so called because when viewed on an oscilloscope, it appears as a rectangle or square. An analog square wave can be created by adding together many frequencies so that each frequency peaks just after the last wave, giving the square shape. See figure 2a for a diagram of a typical binary square wave.
If such a wave were to be transmitted over a voice grade telephone line, only the frequencies in the square wave, between 300 Hz and 3400 Hz would be transmitted, this would distort the signal to such an extent to make it completely unintelligible to a device receiving the signal. Figure 2b shows a binary square wave as it would appear to the receiver.


MODULATION AND DEMODULATION
If the distortion is to be overcome, the rectangular signals of the square wave must behave as ordinary "voice" traffic. That is, it must be converted into the 300 - 3400 Hz range as audio type signals. To do this, a device known as a modem (for MOdulator/DEModulator) is placed between the computer and telephone line at one end and another modem is placed between the telephone line and computer at the other end.
The modem at the originating end takes the stream of digital data waves from the computer and converts or modulates them into audio or "voice" signals capable of surviving the journey without attenuation or appreciable distortion. The modem at the other end takes these audio signals and rebuilds or demodulates the digital square wave pulses into their original form and passes them on to the receiving computer for processing.

THE MODEM
Every modem has two sets of circuits: a modulating circuit to turn the computer data signal into an audio signal and a demodulator to return the audio into a computer data signal. The two circuits are mostly independent of each other and can operate at the same time, if necessary. It is therefore possible and desirable that a computer connected to a modem can both receive and transmit data.

CONNECTING THE MODEM
To get the signals from the modem into the telephone network, the modem is usually directly connected to the telephone jack by the same type of plug used on the end of your telephone. The modem connected to the telephone in this way must be approved by Telecom for installation on their network.
Non-approved modems commonly do not have suitable equipment in them to prevent the transmission of high voltage currents into (or from) the public  telephone network if a fault were to occur. Approved modems have line isolators and cutouts to prevent electrical overload and to prevent the release of voltage to the telephone network.
In the not-so-distant past, modems produced an audible tone through a speaker which was held close to the  mouthpiece of the telephone handset by rubber cups or elastic straps. This type modem is called an acoustic coupler and is now only used where access to a telephone jack is impossible or where the plugs are likely to be incompatible.

MODULATION METHODS
Now here comes the heavy technical part. While it is not important to know how the modem does its magic, I have found it an interesting challenge to attempt to describe the simplest methods in an understandable manner.
There are three common methods of converting the square wave from the computer and turning it into a an audio signal; amplitude modulation, frequency modulation and phase modulation. Figure 4 shows these three methods as they would appear to the receiver.

The first method, amplitude modulation, is where the two audio signals produced by the modem to signify '1's and '0's, are differentiated by the amplitude or strength of a signal frequency, where a burst of power is used to signify the '1'.
In frequency modulation the signals are differentiated completely into two different frequencies, or tones, to signify the '1' or '0'. Frequency modulation is more expensive to implement than amplitude modulation, but the error rate induced by random noise, such as clicks and cross talk, is substantially reduced. This means that data can be transmitted at a faster rate, with no subsequent loss in the quality of the message transmitted.
The third method, phase modulation, is a little more difficult to describe. Consider a constant frequency, or tone, being transmitted down the line. If the phase were to change by 180 degrees, the peaks of the signal wave would become the troughs and vice-versa. This phase change can be use by the modem to modulate the signal into '1's and '0's.
There are two methods commonly used to do this. In the first method, one phase is used to represent the binary '0' and the other to represent the binary '1'. In the second method, a phase change occurs to indicate a transmitted '1', while no change occurs if a '0' is being transmitted.
So, if the modem were to transmit a string of '1's using the first form of phase shift modulation, there would be only two shifts in the phase, once at the start of the '1's and once at the end.  In the second method, there would be as many phase shifts as there are '1's. Figure 5 also shows the two forms of phase shift modulation using a 180 degree shift. Phase shifts can occur at any point on the sine wave but most commonly occur at 0, 90, 180 and 270 degrees. All four phase shifts may be used in some methods of modulation and may be coupled with amplitude modulation to give as many as sixteen different combinations.

Phase shift modulation has the greatest immunity to corruption from line noise, but the circuitry to implement and interpret the phase modulation is correspondingly complex and expensive.
Despite the comparative expense, this type of modulation is the most popular. People have come to demand higher quality and faster transmission rates since modems were first introduced around 15 years ago, so enhancements such as Digital Signal Processing have been invented to cancel or suppress errors caused by the noisy lines which still plague Australia's telephone network.

SPEED OF TRANSMISSION.
Being able to  modulate and demodulate data signals is not sufficient, to get two modems communicating to each other. The two sets of modem / computers must agree on the speed of data transmission. The reason for this may not be immediately apparent.
Imagine two ham radio operators "talking" to each other using Morse code. If one operator is more proficient than the other, that operator would be able to send and receive Morse code at a faster rate than the other. This would cause problems for the slower, less skilled operator. Little snippets of code could be understood and interpreted, but many other words or characters could be missed altogether leaving the message unintelligible. This doesn't happen, of course, because the faster operator would slow down to accommodate the other operators capacities. Now reconsider the modems communicating with each other over the telephone line.
Modems are not all the same, some are cheaper, and use cheaper (and slower) technology, and some are just older, using technology incapable of supporting fast transfer speeds. Most of the newer, faster modems can slow down to match older, slower modems, but the old ones infrequently have the ability to speed up. But all modems are mere automatons, unable to think for themselves, they don't know that they should match the other's speed. They rely on commands from the human operator or computer software to tell the modem to match speeds.
Most newer modems have the ability to detect the other modems speed and auto-select the matching speed but they still must be told to do this one way or another. If the speeds remain unmatched the modems are unable to correctly interpret the other modem's signals and may reject the signal completely, failing to recognise the other modem on the line. Rather like a layman listening to Morse code being transmitted by an expert ham.
The most common speed for data transmission in 1986 was 300/300 bits per second (bps). The slash (/) denotes the separation between the sending speed and the receive speed, which don't have to be the same. This speed corresponds to 30 characters per second (cps), which is a moderate reading speed.  Today, the most common speed is 2400 bps, much faster than many of us can read, but still considered a slow speed for file transfers.
2400 bps modems are giving way quite quickly to 9600 bps modems which can be had for as low as $400.  Faster modems are available, capable of 14,400 bps, which is currently faster than most telephone lines can handle. Of course these fast modems "fall back" to slower speeds when talking with slower modems.

BAUD vs BPS
Just as an aside, you may have read or heard of the term 'baud' as a synonym for bits per second. This is just not correct. Baud rate refers to the number of signal elements which can be transmitted in 1 second. So if data were being transmitted at 600 baud, conventional wisdom would say that 60 characters per second would be transmitted.
Consider, however, that some transfer protocols send four, eight, or even sixteen character bits per signal element using combinations of amplitude, frequency and phase shift modulation. This means that while the 'baud' rate may be only 600, using 4 bits per signal element, the rate of data transfer is 2400 bps, which in turn is roughly equivalent to 240 characters per second.

CHOOSING THE RIGHT SPEED
Potentially, there are literally thousands of speeds and combinations of speeds available for modems to use. If this selection where available considerable confusion would exist for modem users, but there are several protocols or conventions which limit the speed of transfer, the duplex and the modulation method for modem users.
In Australia and Europe we use the standards promulgated by the United Nations organisation, the 'Consultatif Committee d'Internationale Telegraphie et Telephonie' or CCITT.
The most commonly available speed is the V21 (300/300). It is the most common denominator in the modem world. If you have a modem, its lowest speed is most probably 300 and you should be able to communicate with anyone else with a modem, albeit slowly. It will probably continue to be provided in future modems because it is simple to implement.
2400 bps modems are tremendously popular due to a considerable drop in the cost of the complex technologies used to generate such high transfer speeds. In 1986 a mere 1200 bps modem would have cost $900 and a 2400 bps modem about $1800. Today 2400 bps modems cost about $250 and can support all the slower rates I have mentioned here. 300 and 1200 bps only modems are rarely available except as secondhand modems.
The pinnacle of affordable modem technologies reaches 14,400 bps by using combinations of the modulation methods I have discussed earlier. This sort of speed used to be available only to businesses which could afford a special dedicated line. Today, companies like Australia Post, Telecom and BHP use these super-fast modems to transfer great quantities of data throughout Australia. A modem of this sort would cost at least $500 today, although experience shows that prices drop quickly as more people accept and use the technology.

AMERICAN STANDARD CODE FOR INFORMATION INTERCHANGE.
Having a modem successfully connected to another, at the right speed is not quite the end of the story.  Most hobbyists would then want to send or receive some information over those lines.  No comes the problem... with so many different brands of computer out there, how do you make sure that they all use the same "code" to represent letters and numbers?
Let us return to our two communicating ham radio operators. They have agreed on the speed of they are ready to communicate, and they are going to use Morse code as the carrier for their message. Consider again, our modems happily squealing and squawking to each other. They too are using a code, most commonly ASCII, the American Standard Code for Information Interchange, although other codes are used.
The modems, like the ham operators, must sent their information one character at a time by sending 10 signal elements for each character. The Morse code used by our two ham operators is simpler having, at most, 6 elements per character with most alphabetic characters using only 4 elements. In International Morse Code there are less than 50 characters including the uppercase alphabet, digits, and some punctuation characters.
ASCII on the other hand, is a seven bit code which has the ability to handle 128 different characters to accommodate the full range of upper and lowercase characters, digits and punctuation characters. Some computers, which use an Extended ASCII code, are able to offer graphics characters in addition to the Standard ASCII.
Morse code expresses each signal element as one of two durations, one short, and one long. In a computer system, each character transmitted is sent as a series of BInary digiTs (bits), where each bit is represented as either a high or low electronic pulses. By way of comparison then, the letter 'Q' would be expressed as 'dah dah dit dah' in Morse and in ASCII the letter 'Q' could be written as '1010001'. As I have discussed earlier, the '1's and '0's are represented in the computer as pulses of electricity and over the phone lines are modulated for transmission. Either way, there are only two signal levels to represent an unused line.
If the usual state of a line is 'low', then, to signal the other computer that data is about to be transmitted, a 'high' start bit must be sent, otherwise any 'low' bits at the start of the message would be lost. In the same way, one stop bit is added at the end of a character to signify that the whole character has been sent before returning to the idle state.
Serial connections, the ones between modems and computers, rely on fairly precise timing between the receiver and the sender, (it's no good waving to someone unless they are looking), but when the character is framed between a start and a stop bit, any differences in timing are negated.
It is like commencing a voice radio transmission with a call sign and terminating it with an 'over', except that each character is framed by the start and stop bits. Confused? Never mind, as a general, select 8 data bits, one stop bit, and, as I discuss now, no parity when using data services.
Each character is transmitted as eight bits, but as mentioned above, only seven bits are required by the ASCII code to handle 128 different characters of upper and lowercase characters, digits and punctuation marks. The eighth bit is used to transmit binary files which are somewhat circularly defined as files in which all eight bits are required to send a character. Extended ASCII files require such an eight bit code as do most executable program files and the files generated by word processors such as Word, AppleWorks and Word Perfect.
When plain text, (also known as ASCII text), is to be sent, the eighth bit can be used to perform a simple form of error checking called a parity check. If you were to add up all the high bits in a character the result would be an odd or even number. The hereto unused eighth bit can then be set to 1 or 0 to make the sum of the digits always odd or always even.
Hence the names, odd parity and even parity. If parity checking of one form or another is in place, the receiving computer would examine each byte, (a group of eight bits or one character), checking that the sum matches the parity chosen, odd for odd parity and so on. If the sum does not match the parity, the receiving computer alerts the sender to retransmit that character.
Of course, if the line is so corrupt as to generate frequent errors, two complimentary errors may occur in the same byte causing the check to be in order but the byte transmitted will not. This is the main reason why parity checking is not often used, but still remains as the most basic form of error checking. Therefore, if your software requires you to name a parity check, you may choose odd, even or none.
The best bet, in most circumstances, is to choose no parity because your software will govern which, more effective, protocols are used to handle file transfer. Some systems, notably IBM mainframe systems, are extremely finicky over the use of parity, so always follow the service providers recommendations for setup.

PROTOCOLS
Just as there are standards to govern modulation method, the type of transmission (half or full duplex) and the speed of the transfer, there are protocols which support controls and error checking during file transfers. These protocols are not often affected by the speed, code, parity or duplex of the modem but relate to the actual data itself. These protocols work just as well if two computers were directly connected by a special cable as if they were connected by modems over the public switched telephone network.
The most common protocol is used for flow control, called the X-On/X-Off protocol. This protocol is used to permit the receiving computer from overflowing with information when receiving information.
When a stream of characters or a file is being sent, the receiving computer stores all the characters as they arrive in a storage area in its memory called a 'buffer'. When the buffer becomes full, the receiving computer will attempt to store this information on disk in a permanent file.
Since computers prefer to do just one thing at a time, it has to tell the sending computer to stop while it stores the information. In the X-On/X-Off protocol, the receiver sends the X-Off code, a special control character which is the ASCII code 19. Once the buffer has been cleared, the receiver can send the X-On (ASCII 17) code to tell the other computer to resume sending.
The human user of the computer can also control the flow of characters on the computer screen in the same way by sending the special control codes. The ASCII 19 (X-Off) code corresponds to the control-S or <^S> keystroke, that is, hold down the <control> key and press the <S> key. This keystroke effectively pauses the display of characters down the screen. To recommence the display, use the <control Q> keystroke.
The X-On/X-Off protocol only controls flow, it has no error checking facility. The most popular, and readily available protocol to support file transfers is known as XMODEM. The XMODEM protocol is also called the Christensen protocol after its developer Ward Christensen.
This program, and others like it, were developed to reduce the number of errors occurring in text and numerical data files. Unlike the X-On/X-Off protocol it is program dependent and must be run from communications software which supports it. It uses a form of error checking called 'longitudinal redundancy checking'.
This protocol uses no inherent form of data compression and uses tiny "packets" of information so is considered very slow and awkward by modern standards.
In the pure form of this protocol, the sending computer gathers the characters to be sent into blocks of 128 bytes (characters). When it has collected the 128 byte packet, the sending computer transmits a Start Of Header SOH (ASCII 01) character followed by two numbers, corresponding to the number of the block to be sent, to signify the start of data. The 128 bytes are sent immediately followed by a 1 byte checksum.
The checksum is arrived at by the sending computer adding together all the values of the 128 bytes. The computer then composes the checksum from the eight bytes in the accumulated number. The transmitter then waits for the receiver to acknowledge receipt. The receiving computer also calculates the checksum and if the 129th byte doesn't correspond with it's calculation, an error has occurred in the transmission.
The receiver may send either the ACK (ASCII 6) code to confirm that no errors were detected and to cue the next block; or the NAK (ASCII 21) code to signal a transmission error and to cue the retransmission of the bad block. XMODEM also has ways to ensure that the ACK and NAK codes don't become corrupted and the block numbers are checked to make sure that the right one is being transmitted as requested. Overall it is a moderately sophisticated protocol useful for the majority of file transfers under 2400 bps.
YMODEM is a newer protocol which was designed to deal with the needs of users with faster modems, but is suitable for all speeds. YMODEM is similar to XMODEM in that is uses the same sort of error checking but has the added advantage of using 1024 byte blocks. The number of blocks are reduced and as a consequence the overheads of starting and acknowledging blocks is reduced. YMODEM is not as freely available as XMODEM but is still regarded as a desirable alternative.
The protocol I prefer is the ZMODEM protocol, which uses a slightly more effective error checking protocol, called Circular Redundancy Checking (CRC). It also uses an efficient 4096 byte "packet" size and has the ability to transmit file name and filetype information as well. In dire circumstances, the protocol can completely restart a file transmission from a given point if the file transfer initially fails. An all-round useful protocol.
Other protocols abound on varying computers and for different purposes. Some offer data encryption for added security and others promote the transmission of files in batches or groups. The protocol you use will be absolutely dependent on the software you choose and the capabilities of the service to which you are connected.

SUMMARY
This article has introduced some of the basic concepts of modem communications and described the nature of those communications as they might appear to a computer. Some of the limitations of modem communications have also been discussed and an overview presented of how these limitations are overcome by modulation of the data signal. Some of the conventions which govern the actual signal and protocols for encoding the information and correcting data errors have also been introduced.
The revolution that cheap telecommunications has brought to information transfer is still in its youth. As Australia buys more and more into modern technology this revolution will gain more and more in pace. Soon we will see 14,400 bps modems become the norm in telecommunications as the boundary between high cost and high speed recedes.
Some technicians predict the demise of the modem when Australia fully converts to the Integrated Services Digital Network, but this will not occur for some years and even then the modem could conceivably be useful for the wireless transmission of data signals. Until then, modems will still be used to perform most transfers, so the principles I have discussed here will still be practically useful to the novice telecommunicator.

THIS CONTENT COPYRIGHT © 2007, APPLE MACINTOSH USERS' GROUP, SYDNEY
Permission has been obtained to make this material available on the Internet.

Permission is hereby granted for non-profit user groups to republish this content.
PLEASE CREDIT THE AUTHOR AND THE SOURCE: Applecations, publication of the Apple Users' Group, Sydney, Australia

THIS PAGE COPYRIGHT © 2007, ANDREW ROUGHAN